Quality Improvement Opportunities in Prescriber Alert Programs

Battles may be won, but the war against threats to patient safety continues, and time will tell if escalation of the war will reduce casualties. Brent James, MD, the well-known advocate of process improvement, several years ago described himself as a “terrorist” for health care safety. Despite Dr. James’ commitment to escalate the war in an attempt to reduce medical errors, particularly errors that harm, it will be difficult to eliminate adverse drug events (ADEs) entirely because of the myriad of factors that pose risks to patient safety including drug-drug, drug-disease, and drug-patient nuances. But, the certainty of ADEs does not diminish the importance of the mission to reduce patient harm from avoidable ADEs. This mission includes strategic actions, such as proactive avoidance of unnecessary and potentially harmful drug therapy, downward dose adjustment, and even discontinuation of drug therapy as part of high-quality medication therapy management. However, the size of the target is vague because the actual rate of patient harm associated with potentially avoidable ADEs is both controversial and difficult to estimate accurately. Some have argued that even the most frequently cited figures, such as the Institute of Medicine’s estimate that between 44,000 and 98,000 inpatients in the United States die annually as a result of preventable medical errors including ADEs, are either exaggerated by failure to account for patients’ baseline health status or “not well substantiated” because of their reliance on the “highly subjective” opinions of study investigators about which events are avoidable. Additionally, these estimates are widely publicized, represent a large portion of the research on the effects of ADEs, and are commonly cited as evidence of the need for automated prescribing techniques to reduce ADEs throughout the health care system; yet, they do little to inform managed care decision makers about risks to patients in ambulatory care, who represent the vast majority of beneficiaries with public or private insurance. Gurwitz et al. (2003) used a rigorous and labor-intensive methodology, including thorough reviews of patient charts and incident reports, systematic evaluation of all potential ADEs by multiple raters, and assessments of inter-rater reliability, to estimate the incidence, severity, and preventability of ADEs in a cohort of approximately 30,000 predominantly Medicare + Choice (managed care) enrollees who received care in a large multispecialty group practice. During 30,397 person-years of observation over the 12-month period from July 1999 through June 2000, the investigators identified 1,523 events (rate of 50.1 per 1,000 person-years), more than 70% of which resulted in symptoms that persisted for more than 1 day. Of the 1,523 ADEs, 431 (28.3%) were deemed to be “serious” (e.g., fall with fracture, hemorrhage requiring transfusion or hospitalization, delirium, urticaria); 136 (8.9%) were “life-threatening” (e.g., hemorrhage with hypotension, hypoglycemic encephalopathy, acute renal failure); and 11 (0.7%) were fatal, including bleeding, drug toxicity events, anaphylaxis, peptic ulcer, neutropenia, hypoglycemia, and antibiotic-associated diarrhea. Less serious but “significant” events (e.g., nonurticarial skin rash, hemorrhage not requiring transfusion or hospitalization, fall without fracture, and oversedation) constituted 945 (62.0%) of all ADEs. Of 421 ADEs judged as preventable (27.6% of ADEs), 167 (39.7%) were serious, 72 (17.1%) were life-threatening, and 5 (1.2%) were fatal. Thus, of approximately 30,000 elderly enrollees, a total of 244 (0.8%) experienced ADEs that were both preventable and severe during the 12-month follow-up. Still, the overall rate of preventable ADEs identified by Gurwitz et al., 13.8 per 1,000 person-years (approximately 1.4% of enrollees), suggests a problem that should be addressed by managed care. Studies in nonelderly populations support this assessment, albeit not always with a methodology as rigorous of that of Gurwitz et al., usually concluding that there is a small but troublesome rate of potentially preventable ADEs in ambulatory care. Using a pre-implementation versus postimplementation design and multivariate analyses, Devine et al. (2010) studied prescriptions written in a large multispecialty clinic, finding that prior to the implementation of computerized provider order entry (CPOE), 911 of 5,016 (18.2%) prescriptions contained errors, although only 8 of these errors (0.9%, or 0.16% over all prescriptions) caused harm to the patient. After implementation of CPOE, the error rate declined to 423 of 5,153 (8.2%), with 5 of these (1.2%, or 0.10% over all prescriptions) causing harm.


Quality Improvement Opportunities in Prescriber Alert Programs
Frederic R. Curtiss, PhD, RPh, CEBS, and Kathleen A. Fairman, MA B attles may be won, but the war against threats to patient safety continues, and time will tell if escalation of the war will reduce casualties. Brent James, MD, the well-known advocate of process improvement, several years ago described himself as a "terrorist" for health care safety. 1 Despite Dr. James' commitment to escalate the war in an attempt to reduce medical errors, particularly errors that harm, it will be difficult to eliminate adverse drug events (ADEs) entirely because of the myriad of factors that pose risks to patient safety including drug-drug, drug-disease, and drug-patient nuances. 2,3 But, the certainty of ADEs does not diminish the importance of the mission to reduce patient harm from avoidable ADEs. This mission includes strategic actions, such as proactive avoidance of unnecessary and potentially harmful drug therapy, downward dose adjustment, and even discontinuation of drug therapy as part of high-quality medication therapy management. 4,5 However, the size of the target is vague because the actual rate of patient harm associated with potentially avoidable ADEs is both controversial and difficult to estimate accurately. Some have argued that even the most frequently cited figures, such as the Institute of Medicine's estimate that between 44,000 and 98,000 inpatients in the United States die annually as a result of preventable medical errors including ADEs, 6 are either exaggerated by failure to account for patients' baseline health status or "not well substantiated" because of their reliance on the "highly subjective" opinions of study investigators about which events are avoidable. 7,8 Additionally, these estimates are widely publicized, represent a large portion of the research on the effects of ADEs, [9][10][11] and are commonly cited as evidence of the need for automated prescribing techniques to reduce ADEs throughout the health care system; 2 yet, they do little to inform managed care decision makers about risks to patients in ambulatory care, who represent the vast majority of beneficiaries with public or private insurance. Gurwitz et al. (2003) used a rigorous and labor-intensive methodology, including thorough reviews of patient charts and incident reports, systematic evaluation of all potential ADEs by multiple raters, and assessments of inter-rater reliability, to estimate the incidence, severity, and preventability of ADEs in a cohort of approximately 30,000 predominantly Medicare + Choice (managed care) enrollees who received care in a large multispecialty group practice. 5 During 30,397 person-years of observation over the 12-month period from July 1999 through June 2000, the investigators identified 1,523 events (rate of 50.1 per 1,000 person-years), more than 70% of which resulted in symptoms that persisted for more than 1 day. Of the 1,523 ADEs, 431 (28.3%) were deemed to be "serious" (e.g., fall with fracture, hemorrhage requiring transfusion or hospitalization, delirium, urticaria); 136 (8.9%) were "life-threatening" (e.g., hemorrhage with hypotension, hypoglycemic encephalopathy, acute renal failure); and 11 (0.7%) were fatal, including bleeding, drug toxicity events, anaphylaxis, peptic ulcer, neutropenia, hypoglycemia, and antibiotic-associated diarrhea. Less serious but "significant" events (e.g., nonurticarial skin rash, hemorrhage not requiring transfusion or hospitalization, fall without fracture, and oversedation) constituted 945 (62.0%) of all ADEs. Of 421 ADEs judged as preventable (27.6% of ADEs), 167 (39.7%) were serious, 72 (17.1%) were life-threatening, and 5 (1.2%) were fatal. Thus, of approximately 30,000 elderly enrollees, a total of 244 (0.8%) experienced ADEs that were both preventable and severe during the 12-month follow-up.
Still, the overall rate of preventable ADEs identified by Gurwitz et al., 13.8 per 1,000 person-years (approximately 1.4% of enrollees), suggests a problem that should be addressed by managed care. Studies in nonelderly populations support this assessment, albeit not always with a methodology as rigorous of that of Gurwitz et al., usually concluding that there is a small but troublesome rate of potentially preventable ADEs in ambulatory care. 12 Using a pre-implementation versus postimplementation design and multivariate analyses, Devine et al. (2010) studied prescriptions written in a large multispecialty clinic, finding that prior to the implementation of computerized provider order entry (CPOE), 911 of 5,016 (18.2%) prescriptions contained errors, although only 8 of these errors (0.9%, or 0.16% over all prescriptions) caused harm to the patient. 13 After implementation of CPOE, the error rate declined to 423 of 5,153 (8.2%), with 5 of these (1.2%, or 0.10% over all prescriptions) causing harm.

Managed Care Efforts to Avoid ADEs
In the April 2010 issue of JMCP, Feifer and James described an analysis of the geographic variation in drug safety incidents and prescriber responsiveness to a retrospective drug utilization review (DUR) intervention in which an integrated medical and pharmacy claims database was mined using "thousands of algorithms" to generate alerts, which were then transmitted to prescribers by fax or mail. 14 Ollendorf in an accompanying commentary addressed some pertinent questions regarding the takeaway messages from this research. 15 Programs such as that described by Feifer and James are common in managed care, using a variety of methods and data sources, and proponents argue that use of technology to provide patient-and

E D I T O R I A L
alerts, which represented 29.1% of all alerts, and 10.4% of highseverity drug interaction alerts, which represented 61.6% of all alerts. Although there may be some comfort in the positive relationship between the acceptance rate and the severity of the drug interaction alert, the number of alerts in the electronic prescribing system reported by Isaac et al. was large; 6.6% of all attempted prescriptions resulted in at least 1 alert.
The problems with CPOE that were identified by Isaac et al. are not unique. In a systematic review of the literature on the causes and consequences of physician overrides in CPOE alerting systems, Van Der Sijs et al. (2006) found that drug safety alerts were overridden 49%-96% of the time, except for high-severity alerts for overdose, which were overridden 27% of the time. 19 In summarizing the results of 3 studies that examined the reasons for overrides, the primary reason was "alert fatigue caused by poor signal-to-noise ratio because the alert was not serious, was irrelevant, or was shown repeatedly." Additional reasons included the importance of the treatment relative to the risk, physicians' confidence in their own sources of information, or patient resistance. An additional factor was longer alert length, which made the alerts difficult to interpret. Notably, the "alert fatigue" problem appeared to produce potentially serious consequences for patients. In studies with override rates of 57%, 90%, and 80%, respectively, ADEs associated with the overridden alerts were identified in 2.3%, 2.5%, and 5.9%.
Knowledge of alert severity is important not only because it affects physician behavior, but also because it may be associated with the degree to which an ADE can be preventedthat is, higher-severity events are somewhat more likely to be preventable. In the study by Gurwitz et al., only 421 of 1,523 ADEs (27.6%) overall were judged as preventable. However, ADEs were more likely to be judged preventable if they were serious (38.7%), life-threatening (52.9%), or fatal (45.5%). 5 Similarly, using a patient survey (response rate 55%) with subsequent chart review in adult outpatients of 4 adult primary care practices in Boston (2 hospital-based and 2 communitybased), Gandhi et al. found an overall ADE rate of 27.4%, of which 39.2% was either preventable ("due to errors that could have been entirely avoided," 11.0%) or ameliorable ("severity or duration could have been substantially reduced," 28.2%). Of 181 ADEs, 157 (86.7%) were judged as "significant" (less serious), of which 60 (38.2%) were judged as preventable or ameliorable. Of 24 events judged as serious, 11 (45.8%) were preventable or ameliorable.

Drug Safety Alerts as Signal or Noise: The Role of Therapeutic Class
The study by Gurwitz et al. identified a notable relationship between therapeutic class and the avoidability of ADEs. For example, Gurwitz et al. judged as preventable 93 of 203 (45.8%) ADEs associated with diuretics. ADEs associated with drug-specific information to prescribers has the potential to reap both clinical and economic benefits by reducing ADEs and enhancing compliance with health plan formularies. 16,17 However, key information not presented in the report by Feifer and James is of interest to the reader wanting to know how a program of this type contributes to quality improvement.
Foremost is the effect of the large volume of the alerts, reported by Feifer and James as an average of 128 alert messages per 1,000 health plan beneficiaries in 2008. The authors defined an alert "event" as a "potential drug safety issue that generated an alert to 1 or more prescribers." By any measure, this is a lot of messages about drug safety, more than 1.6 million letters or fax communications to prescribers of the approximately 12.6 million health plan members in 1 year alone. Although Feifer and James do not report the distinct number of health plan members affected, the proportion could be as high as 13% in 1 year. Feifer and James indicate in their Limitations section that "prescribers could have received the same alert for the same patient on multiple occasions," making it likely that the actual proportion of patients affected is much less than 13%. More importantly, if the 1.61 million alerts reported by Feifer and James were generated for typical utilization of 0.7 pharmacy claims per member per month or about 8 pharmacy claims per member per year for the 12.6 million members, approximately 1.6% of pharmacy claims generated an alert.
It seems reasonable to ask whether clinicians view alerts provided in this quantity, presumably received from only one of the many pharmacy benefit management companies and health plans with which a typical provider interacts daily, as valuable information or as "noise." To err on the side of minimizing the number of "false negatives" (i.e., failing to provide a drug safety alert for a potentially important ADE) runs the risk of creating so much "false positive" noise that prescribers may not "hear" the true-positive risks to patient safety. The consequent Chicken Little phenomenon (i.e., we can't believe everything that we are told) is no small consideration.

Drug Safety Alerts as Signal or Noise: The Role of Severity
We know from research with electronic prescribing systems that prescribers ignore or override the majority of drug safety alerts. In a study of 233,537 medication safety alerts generated by an electronic prescribing system and provided to 2,872 clinicians over 9 months in 2006, Isaac et al. (2009) found that clinicians overrode 90.8% of the drug interaction alerts and 77.0% of the allergy alerts, accepting only 9.4% of drug alerts overall (allergy alerts represented 1.7% of all drug alerts whereas drug interaction alerts represented 98.3%). 18 The high probability of false-positive alerts for some of the messages is evident in the low rate of acceptance (7.1%) of drug interaction alerts judged to be low severity, although these low-severity drug interaction alerts represented only 7.6% of all safety alerts. Clinicians accepted 7.3% of moderate-severity drug interaction 3% for patient adherence problems. This research by Howard et al. showed that despite a large number of studies of drugrelated hospital admissions, including potentially preventable hospital admissions, more information is needed on the underlying causes of these admissions in order to develop interventions to improve patient safety.
Increasing the Signal While Reducing the Noise: Efforts to Improve Alerting Systems Considered together, the evidence available to date suggests an opportunity to reduce the noise and improve the ratio of truepositive drug alerts of importance to patient safety, continuously improving the quality of alerting systems by using information about severity level or therapeutic class to eliminate altogether or reduce the volume of alerts that are associated with a low rate of drug therapy change. Unfortunately, the report by Feifer and James contains no information about either the volume or response to safety alerts categorized by severity or therapeutic class. Moreover, although Feifer and James acknowledge that they did not adjust their estimates for age, calculating adjustments of this type would appear to yield large benefits in additional information. At the state level, we calculate that 30% of the variance in the alerting rates reported by Feifer and James is explained by mean age. 14,22 For automated systems like that reported by Feifer and James and used by other managed care organizations, more detailed "mining" of the data on drug alert characteristics and outcomes represents a critically important, and to date largely neglected, area for quality improvement and future peer-reviewed research. Weingart et al. (2009a) examined 279,476 drug interaction alerts in an electronic prescribing system from the first half of 2006. An expert panel judged that 402 ADEs may have been prevented, of which 49 were serious, 14 might have resulted in permanent disability, and 3 might have resulted in death. 23 When assessed by severity of harm, the nearly 300,000 alerts may have prevented 39 hospitalizations, or a rate of 1.2 potentially avoidable hospitalizations per 100,000 alerts. Considering all potentially avoidable health care utilization, estimated cost savings totaled $402,619, approximately $1.44 per alert message. However, 331 alerts were needed to prevent 1 ADE. Because 10% of alerts accounted for 60% of ADEs, Weingart et al. concluded that alerts deemed to be of low value should be suppressed.

Can Better Assessment Methods Yield Better Targeting?
In addition to more detailed ongoing monitoring of drug therapy change rates by therapeutic class and severity level, automated alert warning systems could greatly benefit from the findings of studies with better research designs. Particularly important in the Feifer and James analysis is the inability to attribute a change in drug therapy to communication of the alert; instead, changes made within a specified time frame anticoagulants and cardiovascular medications were deemed preventable in 43 of 121 (35.5%) and 103 of 396 (26.0%) cases, respectively. In contrast, ADEs associated with antibiotics and steroids were less often avoidable, in only 13 of 224 (5.8%) and 11 of 80 (13.8%) cases, respectively. 5 Thus, it is perhaps not surprising that physician responsiveness to alert warnings also varies substantially by therapeutic class. In their analysis of prescriber overrides of alert warnings in electronic prescribing, Isaac et al. found that high-severity interactions with low acceptance rates included a methylxanthine such as theophylline prescribed with a cardioselective beta-blocker such as atenolol (acceptance rate 3.4%) or a tetracyclic antidepressant such as mirtazapine prescribed with a serotonin-norepinephrine reuptake inhibitor (SNRI) antidepressant such as venlafaxine (acceptance rate 4.8%). 18 These examples can be compared with high-severity interactions with high alert acceptance rates, such as amiodarone prescribed with either a macrolide such as azithromycin or a quinolone such as levofloxacin, both interactions with prescriber acceptance rates of about 40%.
In addition to its strong association with ADE preventability and provider response, therapeutic class is also important because a few therapeutic classes appear to account for a majority of ADEs, suggesting the possibility of more efficient systems that would generate a lower volume of alerts. Gurwitz et al. found that 5 therapeutic classes-cardiovascular medications, antibiotics, diuretics, nonopioid analgesics, and anticoagulants-were involved in 73.8% of ADEs. Similarly, in a study of preventable drug-related morbidity (PDRM) indicators in 49,658 patients treated in 9 general practices in England, Morris et al. (2004Morris et al. ( , measuring care provided from 1999Morris et al. ( -2002 found that just 4 problems accounted for 59.6% of 507 avoidable PDRMs. 20 These were: (a) use of an oral or topical nonsteroidal anti-inflammatory drug (NSAID) for at least 3 months in a patient with hypertension or congestive heart failure; (b) use of an angiotensin-converting enzyme (ACE) inhibitor without monitoring creatinine level; (c) use of an ACE inhibitor without monitoring potassium level; and (d) use of a hypnotic/anxiolytic with a long half-life.
A systematic review performed by Howard et al. (2006) identified the specific drugs most commonly associated with preventable drug-related hospital admissions caused by either inappropriate prescribing, lack of appropriate monitoring, or patient nonadherence. 21 Only 17 studies were found to be reasonably valid from among 30 relevant studies, and 11 study reports contained sufficient detail to examine the causative drugs. Four drug classes accounted for more than one-half of all drug-related admissions attributed to ADEs or overtreatment: antiplatelets including aspirin when used as an antiplatelet (17.3%), diuretics (16.0%), NSAIDs (12.0%), and anticoagulants (8.9%). Of 1,406 preventable drug-related admissions, 1,263 (approximately 90%) were related to adverse drug reactions or overtreatment, versus 7% for undertreatment and following the issuance of the alert were assumed to be alertdriven. Although a randomized study design could be used to make this assertion with better certainty, withholding alerts in cases of true risk to patient safety raises ethical concerns. Alternatively, it would be a relatively simple matter to construct a comparison group from employer health plans without the drug alerting program and measure the rate of relevant drug therapy changes that occur in the absence of communication of alert messages for the trigger events defined as potential risks to drug safety. The degree of physician responsiveness to drug-related messages is often overstated by studies that fail to account for changes that would have been made even without the intervention. For example, Altavela et al. (2008) found that 23.5% of drug therapy recommendations that would have been made by clinical pharmacists to physicians, but that were concealed from the physicians using a controlled research design, were adopted by the physicians anyway. 24 Also embedded in the significant questions left unanswered by research on the real effects of automated alerting systems on patient safety is rate of receipt of these messages by prescribers. Unlike an electronic prescribing system that requires an action by the prescriber at the point of care, we do not know what proportion of the mail or fax alerts were actually received and interpreted by the prescribers. A more sensitive safety alert system may request return-response affirmation of receipt of the alert and indication of the action taken (e.g., unimportant, no change necessary; discontinued drug X).
Also needed is more information about the quality of these interventions in terms of prescriber perception and satisfaction. Weingart et al. (2009b) found that only 47% of 184 clinicians using an electronic prescribing system were satisfied with the drug interaction and safety alerts. Common problems cited by survey respondents included false-positive alerts triggered by discontinued medications (58%), alerts that failed to account for appropriate drug combinations (46%), and excessive volume of alerts (37%). 25 The challenge to reduce drug-related threats to patient safety is large. The quest for meaningful interventions must be mindful of sensitivity and specificity such that clinicians are required to interpret and take action on high-quality alerts that pose a real threat to patient safety. A great deal of feedback is necessary for such continuous quality improvement to occur. The risk of not doing so is significant, manifest in the proportion of clinicians that may not hear the warnings amid the noise. On the other hand, electronic interventions will soon engulf the practice of medicine, perhaps relegating retrospective DUR safety alerts delivered by snail mail and fax to George Orwell's Room 101. 26 In any case, all that we do in managed care should be guided by a focus on continuous quality improvement, as simple as ABC or PDCA (plan, do, check, act).